Hand-Model-Aware Sign Language Recognition

Hand gestures play a dominant role in the expression of sign language. Current deep-learning based video language recognition (SLR) methods usually follow data-driven paradigm under supervision category label. However, those suffer limited interpretability and may encounter overfitting issue due to data sources. In this paper, we introduce hand prior propose new hand-model-aware framework for isolated SLR with modeling as intermediate representation. We first transform cropped sequence into latent semantic feature. Then model introduces provides mapping from feature compact pose Finally, inference module enhances spatio-temporal representation performs final recognition. Due lack annotation on current datasets, further guide its learning by utilizing multiple weakly-supervised losses constrain spatial temporal consistency. To validate effectiveness our method, perform extensive experiments four benchmark including NMFs-CSL, SLR500, MSASL WLASL. Experimental results demonstrate that method achieves state-of-the-art performance all popular benchmarks notable margin.